Resource Allocation for the French National Grid Initiative

Distribution of resources between different communities in production grids is the combined result of needs and policies: where the users’ needs dictate what is required, resource providers’ policies define how much is offered and how it is offered. From a provider point of view, getting a comprehensive and fair understanding of resources distribution is then a key element for the establishment of any scientific policy, and a prerequisite for delivering a high quality of service to users. The resource allocation model which is currently applied within most national grid initiatives (NGIs) was designed with the needs of the EGEE (Enabling Grids for E-sciencE) projects and should now be revised: NGIs now especially need to assess how resources and services are delivered to their national community, and expose the return on investment for resources delivered to international communities. The French NGI “France Grilles” is currently investigating down this route, trying to define key principles for a national resource allocation strategy that would answer this concern while allowing for the proper definition of service level agreements (SLA) between users, providers and the NGI itself. After looking for clear definitions of who are the communities we are dealing with, we propose to look at how resource allocation is done in other environments such as high performance computing (HPC) and the concepts we could possibly reuse from there while keeping the specificities of the Grid. We then review different use-cases and scenarios before concluding on a proposal which, together with open questions, could constitute a base for a resource allocation strategy for the French national grid. 1 Context and definitions 1.1 Context of current work The EGI-Inspire [1] project started in May 2010, as a continuation of around 6 years of EGEE projects [2]. In this context, the French National Grid Initiative “France Grilles” [3] as emerged as EGI’s partner for federating and operating grid resources in France. Within EGI operational context [4], allocation of 2 Gilles Mathieu, Hélène Cordier resources to grid user has changed scope, since national grids are now privileged interlocutors and interfaces between users and providers. This work is a preliminary reflection on the topic of resource allocation, and a possible basis for establishing policies and procedures in the medium term specific to France’s context and based on international collaboration. 1.2 Definition of “user communities” In this paper, “user community” is used to represent a logical grouping of users that can be seen as a unique interlocutor for all other actors. In our Grid context, a typical example of a user community is a Virtual Organization (VO), but this can also be extended to Virtual Research Communities (VRCs) or a specific scientific community federated around a given project. 1.3 Definition of “resource providers” and “service providers” Resource providers are the entities that provide user communities with access to computing and storage resources. They are grid resource centres or “sites” as described in EGI operational architecture definition [4]. Service providers are entities offering services that can be technical – e.g. core VO services, monitoring tools etc. – or not – e.g. support or expertise. France Grilles places itself both as a service provider and an operation centre as defined in [4]. 1.4 Definition of “resource allocation” We consider “resource allocation” as a process involving different partners with the aim to provide resources and services to Grid users. The result is actual resources and services being provided, but also agreements being set up between providers and consumers. Involved actors are user communities, resource providers and hyper-structures such as Grid Infrastructures. “Allocation” is understood in this context, and should not be interpreted as “reservation”. 2 Identified needs and goals 2.1 Improve service delivery to the French Community The French Grid Infrastructure “France Grilles”, like most of its counterparts, has been set up to answer specific scientific needs according to a national scientific policy. Overall supervision on resource allocation and distribution is highly desirable and should be done in respect to this. An allocation policy is Resource Allocation for the French National Grid Initiative 3 clearly needed to get a comprehensive and fair understanding of resource distribution. If there is a need to re-equilibrate this distribution of resources between different communities (VOs, VRCs, projects, etc.), this should be done according both to the needs and the overall scientific policy. This is an essential contribution to a better quality of service delivered to our users. 2.2 Measure what is done There has been no clear resource allocation policy so far within France Grilles: the current resource allocation model which was designed with the needs of EGEE projects should be revised to ensure the visibility and sustainability of the French Grid Infrastructure. Beyond that, there is a clear need of accountability. Especially, France Grilles needs to be able to: ‐ Assess how resources and services are delivered to the French community; ‐ Justify that resources delivered to international communities are not wasted, and that there is a return on investment. 3 Inspiration from existing resource allocation mechanisms 3.1 High Performance Computing (HPC) world Resource allocation is an important aspect of all computing infrastructures, and High Performance Computing is no exception. In this particular domain, resource allocation is based on scientific evaluation, through a priori (e.g. evaluation of answers to a call for proposals) and a posteriori (regular review of supported demands) analyses. This is how resources are allocated in the GENCI [5] project, as explained as early as 2007 in [6] and reflected in the yearly activity report from 2009 onwards [7]. These concepts could be adapted to the Grid context. However this has to be done with care to take into account Grid specificities such as “free” access to resources and the absence of the concept of resource reservation. 3.2 Worldwide LHC Computing Grid (WLCG) The WLCG [8] resource allocation model is based on a principle of pledges: to answer LHC experiments’ needs, participating resource providers offer amounts of computing resources under the form of pledges under the supervision of a “Resource Scrutiny Group” on a yearly basis [9]. WLCG being the biggest user of France Grilles resources, it is utterly important to take this procedure into account. 4 Gilles Mathieu, Hélène Cordier 3.3 Other national grids Discussions and collaborations with the polish NGI PL-Grid [10] have led to sharing ideas and concepts about resource allocation at a national level. PLGrid model is a resource allocation centric model [11] which makes extensive use of an SLA management tool, the Grid Bazaar [12]. Interactions with our Polish colleagues have already produced some of the ideas described in this paper. The use of a bazaar-like tool is equally one of the tracks we could follow in the future. 4 Definition of the strategy